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Abstract 

Teacher and student behaviors in the classroom have been linked to student achievement. 
The hands-on, real world experiences which students are offered through career and technical 
education courses provide an opportunity for agricultural education to make contributions to 
student achievement. The purpose of this study was to compare engaged time during traditional 
instruction and during the instruction of an inquiry> based curriculum (CASE). The target 
population for this study consisted of students enrolled in animal science courses taught by 
secondary> agriculture teachers. The study employed a quasi-experimental, static-group 
comparison design. Nine CASE certified teachers represented the treatment group, which were 
matched with nine traditional agriculture teachers on selected criteria. Student engagement was 
measured using the Behavioral Observation of Students in Schools. Matched pairs t-tests were 
used to compare the CASE group and the traditional group on student engagement. Students in 
the CASE group were found to spend significantly more time actively engaged than those in 
traditional agriculture courses. From the findings it was concluded that the CASE curriculum 
and professional development can impact the active engagement of students in the classroom and 
potentially affect student achievement. 

Keywords: Student Engagement, Curriculum for Agricultural Science Education, CASE, BOSS, 
Engaged Time, Off-Task Behavior 

The national emphasis that is placed on student performance in science, technology, 
engineering and math (STEM) and the requirements of No Child Left Behind have initiated the 
development of novel teaching strategies, teacher training programs, and curricular resource 
organizations such as Curriculum for Agricultural Science Education (CASE). In addition, career 
and technical education (CTE) courses have the potential to integrate coursework, work-based 
learning experiences and hands-on experience which allow students to develop competencies, 
skills, and attitudes for success beyond high school (New York City, 2008). 

With more than 510,000 students enrolled in agricultural education courses, programs 
have the opportunity to play a vital role in improving performance in STEM areas. While the 
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current national level of science integration in agriculture curricula is unknown, studies have 
shown core curriculum integration can have positive effects on student achievement (Parr, 
Edwards, & Leising, 2009). Furthermore, there has been a great deal of effort to identify the 
critical instructional variables affecting students’ mastery of skills (Shapiro, 2011a). The concept 
of student engagement and academic engaged time emerged as one of these important factors 
from Carroll’s (1963) model of school learning. This model, shown in Figure 1, makes the 
hypothesis that, if all other things are held equal, learning can be seen as a simple function of the 
time a student is actively engaged in learning. 

(time actually spent\ 

Degree of learning = f --—-- 

\ time needed ) 

Figure 1. Carroll’s Model of School Learning 

Carroll (1963) identified learning as a function of five factors; aptitude, ability, 
perseverance, opportunity to learn, and the quality of instruction. Other researchers have built 
upon the model and agree student learning can be thought of as a function of the amount of time 
students are actively engaged in instruction (Bloom, 1974; Bloom, 1976; Denham & Lieberman, 
1980; Stanley & Greenwood, 1981; Wiley & Flamischfenger, 1974). Fredricks, Blumenfeld, and 
Paris (2004) reported on several studies that demonstrated a positive correlation between 
behavioral engagement and student achievement across all grade levels. Other research has 
reported fewer discipline and behavior issues in classrooms with higher levels of engagement 
(Finn & Rock, 1997). The results of these studies suggest academic engaged time is an important 
variable to consider when investigating student academic performance. 

There are two factors that account for time spent learning: time allowed and perseverance 
of the student. The National Education Commission on Time and Learning (1994) described this 
as net instructional time, which accounts for the time students are actually receiving instruction 
after deducting the time spent on non-instructional activities. Since this report, there have been 
few changes to the length of the school day or year (NECTL, 1994). Flowever, the commission 
believed the way time is used can have an impact on student achievement. The effect of time 
allowed has been studied by several researchers (Caldwell, Fluitt & Graeber, 1982; Goodman, 
1990; Karweit, 1983; Karweit & Slavin, 1981). They supported the findings of Dewalt and 
Rodwell (1988), whose research reinforced the necessity of employing sound instructional 
delivery techniques to improve achievement in schools. This notion was further supported by 
Mulholland and Cepello (2006) who found that high quality curriculum, designed to improve 
teacher candidates’ skills in the classroom, had positive effects on engagement and achievement. 

Associated with this notion is the concept of perseverance, which is described as the 
amount of time the student is willing to engage in the learning activity or the percentage of class 
time actively working in a subject area (Caldwell et al., 1982). It is believed, learners require 
different amounts of time to learn or master new concepts. Schools generally provide the 
opportunity for instruction and practice of these concepts, but there are no guarantees students 
will take advantage of these opportunities. Carroll (1963) posited all students have a point at 
which they are no longer willing to learn and recognized that many different variables can affect 
this point. Students’ motivation to learn plays a major role in classroom perseverance. Some 
students might begin to learn and then lose interest while others might be intimidated by the task 
and never begin. As a result of their findings, Berliner (1980) encouraged teachers to measure 
understanding, increase clarity, and involve the students to improve engagement instead of using 
discipline. 

Given the development of knowledge surrounding the effects of time in schools, 
researchers and policymakers have been increasingly focused on student engagement as the key 
to addressing low achievement, student boredom, and high dropout rates (Fredricks et al., 2004). 
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In order to continue to advance the knowledge on engagement, it is important there be a common 
understanding of the terms academic, cognitive, intellectual, institutional, emotional, behavioral, 
social, and psychological engagement, as found in the literature (Taylor & Parsons, 2011). 
Fredricks et al. (2004) summarized this research into three clearly defined types of engagement. 
These three types—behavioral, emotional, and cognitive—are considered to be three unique types 
of engagement; yet, they have also been used together as a meta-construct. 

Behavioral engagement is defined in several ways. The first definition involves positive 
conduct in schools such as following classroom rules and a lack of disruptive school behavior 
(Finn, 1993; Finn, Pannozzo, & Voelkl, 1995; Finn & Rock, 1997). Other definitions involve 
student participation in school sponsored activities (Finn, 1993; Finn et al., 1995). Finally, and 
most applicable to this study, behavior engagement involves students’ involvement in learning 
and academics such as effort, persistence, and concentration (Carroll 1963; Brich & Ladd, 1997; 
Skinner & Belmont, 1993). Academic engagement is a concept nested below behavioral 
engagement, and is referred to as a combination of classroom behaviors indicating a student is 
involved in instruction. 

Emotional engagement refers to the affective component of student engagement. 
Researchers have defined and measured this as interest, boredom, anxiety, and happiness related 
to learning (Connell & Wellborn, 1991; Skinner & Belmont, 1993). Students’ connection to the 
school or courses is a large component. Emotional engagement has been largely studied in 
situations where students are in high-risk environments. Fredricks et al. (2004) reported the 
majority of studies of emotional engagement utilize student self-report measures that include 
items “related to school, schoolwork and the people at school” (p. 66). 

Cognitive engagement stems from the literature on school engagement involving self- 
regulated learning. Self-regulated learning can be thought of as rewarding or punishing one’s 
own behavior based on personal goals (Slavin, 2009). Researchers have explored this concept 
using a variety of methods. Appleton, Christenson, Kim, and Reschly (2006) surveyed 
participants in Midwest urban school districts using the Student Engagement Instrument (SEI). 
The SEI was developed by the researchers using an extensive literature review of studies 
discussing cognitive and psychological engagement. The final version of the SEI contained 30 
items measuring cognitive engagement and 26 items measuring psychological engagement. 

Results from Appleton et al. (2006) revealed the items on the SEI were valid 
measurements of students’ cognitive engagement in the classroom; however, the SEI tells 
practitioners very little about what is actually happening in the classroom. These measures are 
more valuable in identifying students who are at risk for educational failure (Appleton et al, 
2006). 

Meece, Blumenfeld, and Hoyle (1988) conducted an early study on students’ cognitive 
engagement. They found students’ goal orientations, or attitude toward learning, had an impact 
on the level of engagement. “Students who reported greater intrinsic motivation to learn placed a 
stronger emphasis on educational goals and learning” (Meece et al., 1988, p. 521). Another 
interesting finding from their study indicated students’ academic ability did not aid in the 
prediction of students cognitive engagement. In 2007, researchers revisited the Carroll Model of 
School Learning, calling for policy makers to provide more time for the core academic subjects as 
a way to improve lagging scores (Resnick, 2007); however, they also recognized that spending 
more time in the classroom was not enough. Students must be actively engaged in the instruction. 
This “rate of engagement is influenced by how well structured the teaching is with respect to 
individual students” (Resnick, 2007, p. 1). 

Another important aspect to consider is the impact of time on deep and robust learning of 
the subject matter. Researchers contend that as teachers are required to teach more and more 
content, their natural reaction is to move through the material more quickly (Resnick, 2007). 
Clark & Linn (2003) found when students received less instructional time for the same content, 
their ability to demonstrate deeper understanding in essay question responses fell dramatically. 
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Time in school and engaged time is only one piece of the puzzle. The measurement of 
time does not mirror the cognitive processes in a student’s head. “All that we can say with some 
certainty is that any learning that happens to occur does require time” (Carroll, 1989, p. 27). 
Engaging students in the classroom is still a challenge for educators. In a study of cognitive 
engagement, researchers found students did not believe they were responsible for driving their 
own level of engagement. They consider this action to be the responsibility of the instructor 
(Erry & Wood, 2011). 


Purpose and Objectives 

Student engagement has become a valid indicator of institutional excellence among 
researchers and practitioners (Axelson & Flick, 2011). Additionally, the li nk between student 
achievement and engagement has been explored and supported by a number of researchers 
(Connell, Spencer, & Aber, 1994; Fredricks et al., 2004; Skinner, Wellborn, & Connell, 1990). 
This study sought to compare engaged time during traditional instruction and during the 
instruction of an inquiry based curriculum (CASE). 

The purpose of this study was to investigate the academic engagement of students 
enrolled in animal science courses taught with the CASE curriculum and teacher selected 
curriculum. The following research objective and hull hypotheses were used to guide the study: 

1. Compare the engaged time and off-task behavior of students enrolled in traditional 

animal science courses and students enrolled in CASE animal science courses. 

1. H 0 : There is no difference in the total engaged time of students in traditional and 
CASE animal science courses (H ote t: Ptraditionai = Pcase). 

2. H 0 : There is no difference in the active engaged time of students in traditional 
and CASE animal science courses (H 0ae t: Ptraditionai = Pcase). 

3. H 0 : There is no difference in the passive engaged time of students in traditional 
and CASE animal science courses (H 0pe t: Ptraditionai = Pcase). 

4. H 0 : There is no difference in the off-task behavior time of students in traditional 
and CASE animal science courses (H 0o ft: Ptraditionai = Pcase). 

5. H 0 : There is no difference in the off-task motor time of students in traditional and 
CASE animal science courses (H 0o ft- m : Ptraditionai = Pcase). 

6. H 0 : There is no difference in the off-task verbal time of students in traditional 
and CASE animal science courses (H 0o ft- V : Ptraditionai = Pcase). 

7. H 0 : There is no difference in the off-task passive time of students in traditional 
and CASE animal science courses (H 0o ft- P : Ptraditionai = Pcase). 

Methods and Procedures 

This study employed a quasi-experimental, static-group comparison design. Gall, Gall, 
and Borg (2007) define quasi-experimental design as a study lacking random assignment to 
groups. “This type of experiment, if carefully designed, yields useful knowledge. However, you 
should be aware of the special problems that can arise when individuals are not assigned 
randomly to groups” (p. 416). The static-group comparison design is characterized by the use of 
treatment and control groups, and investigating the differences between them through the use of a 
post-test. 

Population 

Nine CASE Agricultural Science-Animal certified teachers were identified to participate 
in the study. Each CASE teacher was then matched with an agriculture teacher who was not 
CASE certified. Matches were selected based on location, years of experience, and school size. 
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Students, of the selected teachers, who were enrolled in animal science classes, were asked to 
participate in this study. Students were taught by their teacher using one of two different 
curriculum options, CASE or the teacher’s previously selected curriculum. Since the teachers 
and students were part of intact classroom groups, random assignment was not used at any point 
in the study to assign a specific curriculum to teachers and students. Gall et al. (2007) identify 
intact groups as a set of individuals who must be treated as a defined group instead of unique 
individuals. 

Students received information about the study and the need for class recordings. Assent 
and consent forms were required to participate in the study. Students who did not return both 
assent and consent were not included. Teachers were instructed to have the non-participants sit 
outside of the camera view. All protocols were approved by the Human Research Protection 
Program at Texas Tech University. 

Design 


While the independent variable in this study was the type of animal science curriculum 
the agriscience teachers used in their classroom, the dependent variables were academic engaged 
time and off-task behaviors of students. Shapiro (2011b) defined academic engagement as a 
combination of two subcategories: active and passive engaged time. Active engaged time is 
defined as “those times when the student is actively attending to the assigned work” (p. 42). 
Passive engaged time is defined as “those times when the student is passively attending to 
assigned work” (p. 43). 

The main threat to a study of this type is the challenge of attributing the differences 
between the groups to the experimental conditions (Gall et al., 2007). Campbell and Stanley 
(1963) reported it is possible the differences between the groups might have occurred even 
without treatment. The use of matched pairs is one way of improving the internal validity of the 
study. Gall et al. (2007) identify internal validity as “the extent to which extraneous variables 
have been controlled by the researcher, so that any observed effects can be attributed solely to the 
treatment variable” (p. 642). Threats to internal validity include history, maturation, testing, 
instrumentation, statistical regression, selection, and mortality (Campbell & Stanley, 1963). 

Treatments 

The CASE curriculum was developed in 2007 by the National Council for Agricultural 
Education as part of an effort to implement a national curriculum for secondary agricultural 
education. The curriculum provides challenging and integrated lessons to enhance the rigor and 
relevance of agriculture, food, and natural resources (AFNR) subject matter. The primary goal of 
CASE is to improve math and science education by creating a context for student learning 
through agricultural education courses. CASE curriculum aligns the AFNR content standards 
with those of science, mathematics, and English. This study only investigated the effects of the 
Agricultural Science - Animal (ASA) curriculum. 

The CASE model differs from the traditional method of instruction in several ways. 
CASE works to ensure quality teaching by providing professional development for teachers 
before they use the curriculum. The CASE Institute is a professional development workshop 
requiring 80 hours of intense training for each course CASE has developed. 

Teachers using traditional animal science curriculum were used as the control group for 
this study. Traditional curriculum was defined as any curriculum or instructional strategies other 
than the official CASE curriculum. Newcomb, McCracken, and Warmbrod (1993) explain 
curriculum in traditional agricultural education classrooms can be very different from program to 
program. The content is often heavily influenced by the educational philosophy of the agriculture 
teacher, the expectations of the community, and the educational resources available to the teacher. 
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Various textbooks and curriculum guides are available to agriculture teachers across the country; 
however, these are often developed for a specific state or region and must be adapted by the 
individual teacher (Newcomb et al., 1993). 

Instrumentation 

For the purposes of this study, the Behavioral Observation of Students in Schools (BOSS) 
was used in order to collect the needed data. The BOSS was used to describe the academic 
engagement and off-task behavior of students through the use of direct observation (Shapiro, 
2011b). The BOSS consists of five categories of student behavior: active engaged time, passive 
engaged time, off-task motor, off-task verbal, and off-task passive. The BOSS was designed to 
observe a target student’s behavior and compare it to the behaviors displayed by their peers. For 
this study, the BOSS was used in a modified manner to describe the behavior of all students in the 
classroom as opposed to an individual student. Studies have employed the BOSS to observe the 
academic engagement of entire classrooms instead of only comparing a target student to their 
peers (Mautone, DuPaul, & Jitendra, 2005; McQuillan, DuPaul, Shapiro, & Cole, 1996; Vile 
Junod, DuPaul, Jitendra, Volpe, & Cleary, 2006). 

The Academic Skills Problems Workbook (Shapiro, 2011b) was used to guide the data 
collection process with the BOSS. Momentary time sampling using 15-second intervals was 
employed for data collection. Active engaged time (AET) and passive engaged time (PET) are 
scored using momentary time sampling at the beginning of each 15-second interval. Throughout 
the remainder of each interval, the partial interval method was used to record the three types of 
off-task behavior. This means students could only be marked as engaged if the behavior was 
observed at the beginning of each interval, while off-task behaviors were recorded at any time. It 
is also important to note multiple behaviors could be marked per interval. For example, a student 
who is taking notes can be actively engaged at the beginning of an interval, but then be marked 
off-task verbal for making an unrelated comment to another student during that same interval. 

A student was randomly selected to serve as the first student to be observed in each class 
session. After the first student was observed for the 15-second interval, the observation was 
conducted on the second student in the class for their 15-second interval. The observation was 
rotated around the room until all of the participating students had been observed and then the 
rotation started again with the first randomly selected student. The rotation continued until the 
course had ended for the day. 

Shapiro’s (2011b) descriptions of student behavior were used to guide the coding 
process. AET is defined as those times when the student is actively attending to the assigned 
work. Examples include reading aloud, writing, raising a hand, or talking to a teacher or peer 
about the assigned material. PET is defined as those times when the student is passively 
attending to assigned work. Examples include listening to a lecture, looking at a worksheet, 
reading assigned material silently, or listening to the teacher or a peer about assigned work. 

Non-engagement is categorized into three individual types of behavior: off-task motor, 
off-task verbal, and off-task passive. Off-task motor behaviors (OFT-M) are defined as any 
instance of motor activity that is not directly associated with the assigned academic task. 
Examples of OFT-M include any out-of-seat actions not assigned, aimlessly flipping pages of a 
book, manipulating objects not related to the academic task, touching other students or objects, 
and fidgeting in one’s seat for at least three consecutive seconds. Off-task verbal (OFT-V) 
behaviors are defined as any audible expressions not permitted or not related to the assigned task. 
Example of OFT-V include whistling, humming, talking to others, making unauthorized 
comments or remarks, and calling out answers when not asked by the teacher. Finally, off-task 
passive (OFT-P) behaviors are defined as times when a student is passively not attending to an 
assigned academic activity for at least three consecutive seconds. Examples include sitting 
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quietly during unassigned time, looking around the room, staring out the window, or passively 
listening to other students talk about issues unrelated to assignments. 

Validity and Reliability 

Gall et al. (2007) discuss the concept of validity and reliability through the lens of 
classical test theory. Classical test theory makes the assumption that each individual has a true 
score for a given measurement. The score actually observed is a product of the individual’s true 
score and a certain amount of measurement error. The validity and reliability of those tests can 
be used to mitigate the amount of error in the observed scores of the participants. 

While the BOSS was developed by Shapiro (2011b) to assess the academic engaged time 
of students in the classroom, currently there is no published data on the convergent validity of the 
BOSS as a measure of classroom engagement. However, it has been shown that the BOSS is 
successful in discriminating students with ADHD from their nondisabled peers, demonstrating 
good discriminant validity (Volpe, DiPerna, Hintze, & Shapiro, 2005). Researchers have used the 
instrument in a number of studies to determine the academic engagement of students in the 
classroom and deemed it valid (Hintze & Matthews, 2004; Mautone, DuPaul, & Jitendra, 2005; 
McQuillan, DuPaul, Shapiro, & Cole, 1996; Spanjers, Burns, & Wagner, 2008). 

For this study, one researcher observed all of the videos and scored the BOSS. When one 
observer scores all videos, the potential for error is compound across all of the data collection. 
Intra-observer reliability is the estimate of one person’s consistency of scoring. The intra¬ 
observer reliability was calculated prior to beginning data collection through the use of a pilot 
test. A video of a college class was viewed by the rater who scored the BOSS and the same video 
was viewed and scored again two weeks later. The researcher’s intra-observer reliability for the 
video was calculated as .98 (Pearson correlation coefficient). 

Data Collection 

Participating teachers were sent a Flip video camera, 16 GB flash drive, and detailed 
directions for recording their classes. Teachers were instructed to record three class periods of 
their animal science course and then return the equipment and videos using a prepaid mailing 
label. There was no set recording schedule for the videos in order to allow flexibility for busy 
schedules. 

The use of video cameras to conduct observation in the classroom can create ecological 
validity concerns. Ecological validity refers to the ability to generalize conclusions due to 
environmental conditions created by the researcher. The Hawthorne effect refers to a situation 
where the experimental conditions, such as the presence of a video camera, can change behavior 
of subjects (Gall et al., 2007). If teachers or students change their behavior as a result of the 
presence of the video camera, the findings might not be representative of the environment when 
the camera is not present. In order to mitigate the Hawthorne effects, the researcher was never 
present in the classroom and participants were reminded that normal classroom behavior should 
be recorded. 

Frank, Juslin, and Harrigan (2005) reported that studies using video equipment can 
maintain good levels of ecological validity. They recommend turning off the red recording light 
and placing the camera out of the line of sight, which were both used in this study. The use of 
repeated recordings also increases the ecological validity by wearing away the novelty of the 
camera (Gall et al., 2007). Babad (2005) stated ratings of videotaped classrooms are less biased 
than self-report measures of students, further supporting the use of video recording in this study. 

Upon receiving the videos from teachers, the researcher completed the observational 
instrument selected for the study according to Shapiro’s (2011b) guidelines for the instrument. 
An audible timing device notified the researcher of each new interval. In addition to the audible 
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signal, the timer kept track of the number of intervals so the researcher was aware of the specific 
interval of each observation. Data were entered into Qualtrics, an online survey tool, to eliminate 
any inconsistencies in data entry. Analysis of data was conducted using the Statistical Package 
for the Social Sciences (SPSS) version 18.0 computer program for windows. The alpha level for 
all statistical tests was established a priori at .05. 

Data Analysis 

Frequency counts and percentages were used to describe the occurrence of each type of 
student behavior: AET, PET, OFT-M, OFT-V, and OFT-P. Total academic engaged time was 
calculated by adding the total number of occurrences of AET and PET. Total time off-task was 
calculated by adding the total number of occurrences of OFT-M, OFT-V, and OFT-P. Percentage 
of total academic engaged time and time off-task were also calculated. Each type of student 
behavior, engaged and off-task, is an independent observation, resulting in an independent total 
percentage for both categories. As a result, hypothesizes were developed for all seven categories 
of data. 

In order to test the null hypothesizes, paired samples t-test were calculated to compare 
data within total engaged time, total off-task time, and each of the five types of student behavior. 
Calculations of effect size were conducted using a Microsoft Excel" spreadsheet designed 
specifically for calculating Cohen’s d from t-tests (Thalheimer & Cook, 2002). 

Results and Findings 


Descriptive Data 

In order to describe students’ on-task behavior, the researcher totaled the occurrences of 
AET and PET from the three observations. This total was divided by the total number of 
intervals observed to find the percent of time-on-task. Students in the traditional courses 
displayed on-task behavior during an average of 67.23% (SD = 5.68) of the intervals, while 
CASE students displayed on-task behavior during an average of 62.45% (SD = 7.52) of the 
intervals (see Tablel). Similarly, the percentage of off-task behavior was calculated by 
combining the percentage of intervals OFT-M, OFT-V, and OFT-P observed and dividing by the 
total number of intervals. The researcher found students in the traditional courses were off-task 
for an average of 40.46% (SD = 5.19) of intervals, while student in the CASE courses were off- 
task 44.03% (SD = 7.86) of intervals. 
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Table 1 


Percent of Engaged and Off-Task Behaviors for Students (n =18 9) 


Behavior 

M 

SD 

Range 

(Min-Max) 

Traditional (n =9) 

AET 

20.09 

8.97 

5.34-36.72 

PET 

47.14 

8.17 

36.17-58.02 

Total Engaged 

67.23 

5.68 

58.81-77.54 

OFT-M 

20.80 

4.15 

17.62-30.77 

OFT-V 

14.67 

5.58 

8.33-24.61 

OFT-P 

4.98 

2.04 

1.96-8.40 

Total Off-Task 

40.46 

5.19 

32.80-48.15 

CASE (n = 9) 

AET 

30.25 

7.12 

19.10-37.46 

PET 

32.24 

8.54 

19.31-43.72 

Total Engaged 

62.45 

7.52 

46.55-71.28 

OFT-M 

23.48 

5.07 

17.96-32.07 

OFT-V 

16.68 

4.48 

10.24-25.86 

OFT-P 

3.87 

2.31 

1.22-8.63 

Total Off-Task 

44.03 

7.86 

35.21-61.55 


Note. Engaged and off-task are independent of each other and will not total 100% if combined. 


Students taught by traditional agriculture teachers spent the largest percentage of 
intervals passively engaged (M = 47.14, SD = 8.97), which ranged from 36.17 to 58.02%. This 
was followed by off-task motor, with a mean of 20.80% (SD = 4.15) of the intervals. Students in 
this group spent the least amount of intervals passively off-task, with only 4.98% (SD = 2.04) of 
intervals. 

Students taught by CASE certified teachers spent the largest percentage of intervals 
passively engaged (M = 32.24, SD = 8.54). This was followed closely by active engaged time, 
which had a mean of 30.25% (SD = 7.12). Students in this group spent the least amount of time 
passively off-task, with only 3.87% (SD = 2.31) of intervals in which this behavior was observed. 

Hypothesis Testing 

The first null hypothesis stated there was no difference in total engaged time between the 
two groups (see Table 2). No differences were found between traditional and CASE students’ 
time on-task (t = 2.09, p = .07). Null hypothesis one was accepted and therefore was held as 
tenable. However the difference represents a large effect size (d = .76) 

Table 2 


Comparison of Student Time On-task (n = 18) 


Group 

n 

M 

SD 

t 

P 

Traditional 

9 

67.23 

5.68 

2.09 

.07 

CASE 

9 

62.45 

7.52 




The second null hypothesis stated there will be no difference in the active engaged time 
of students in courses taught by traditional agriculture teachers and those taught by CASE 
certified teachers (see Table 3). The results indicated that students in CASE courses (M= 30.22, 
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SD = 7.12) spent significantly more time ( t = -2.50, p = .04) actively engaged than those in 
traditional courses (M- 20.09, SD = 8.97). This difference represents a large practical effect (d = 
1.25). The null hypothesis was rejected and therefore not held as tenable. 

Table 3 


Comparison of Student Engaged Time (n = 18) 



Traditional (n = 9) 

CASE (n 

^9} 



Behavior 

M 

SD 

M 

SD 

t 

P 

AET 

20.09 

8.97 

30.22 

7.12 

-2.50 

.04* 

PET 

47.14 

8.17 

32.24 

8.54 

4.11 

.01* 


The third null hypothesis stated there will be no difference in the passive engaged time of 
students taught by traditional agriculture teachers and those taught by CASE certified teachers. 
The results indicated students in traditional courses (M = 47.14, SD = 8.17) spent significantly 
more time (t = 4.11, p = .01) passively engaged than those in CASE courses (M = 32.24, SD = 
8.54), representing a large practical effect (d = 1.78). The null hypothesis was rejected and 
therefore not held as tenable. 

The fourth null hypothesis stated there will be no difference in the total time off-task of 
students in courses taught by traditional agriculture teachers and those taught by CASE certified 
teachers (see Table 4). The results indicated there was no difference (t = - 1.33, p = .22) between 
students in traditional courses (M= 40.46, SD = 5.19) and those in CASE courses (M = 44.03, SD 
= 7.86). Although the difference was not significant, it did represent a medium-sized effect (d = 
0.53). The null hypothesis was accepted and therefore held as tenable. 

Table 4 


Comparison of Student Off-task Behavior (n = 18) 


Group 

n 

M 

SD 

t 

P 

Traditional 

9 

40.46 

5.19 

-1.33 

.22 

CASE 

9 

44.03 

7.86 




The fifth null hypothesis stated there will be no difference in the off-task motor time of 
students in courses taught by traditional agriculture teachers and those taught by CASE certified 
teachers (see Table 5). The results indicated there was no difference (t = -1.18,/> = .27) between 
students in traditional courses (M= 20.80, SD = 4.15) and those in CASE courses (M = 23.48, SD 
= 5.07). The difference represented a large-sized effect (d = 0.73). The null hypothesis was 
accepted and therefore held as tenable. 

Table 5 

Comparison of Student Off-task Behavior (n = 18) 


Traditional (n = 9) CASE (n = 9) 

Behavior _M_ SD _ M_ _ SD _ t _ p_ 


OFT-M 

20.80 

4.15 

23.48 

5.07 

-1.18 

.27 

OFT-V 

14.67 

5.58 

16.68 

4.48 

-1.17 

.28 

OFT-P 

4.98 

2.04 

3.87 

2.30 

1.76 

.12 
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The sixth null hypothesis stated there will be no difference in the off-task verbal time of 
students in courses taught by traditional agriculture teachers and those taught by CASE certified 
teachers. The results indicated there was no difference (t = -1.17, p = .28) between students in 
traditional courses (M = 14.67, SD = 5.58) and those in CASE courses (M = 16.68, SD = 4.48). 
The difference was found to be a small sized effect (d = 0.40). The null hypothesis was accepted 
and therefore held as tenable. 

The seventh and final null hypothesis stated there will be no difference in the off-task 
passive time of students in courses taught by traditional agriculture teachers and those taught by 
CASE certified teachers. The results indicated there was no difference (t = 1.76, p = .12) between 
students in traditional courses (M= 4.98, SD = 2.04) and those in CASE courses (M= 3.87, SD = 
2.30). The null hypothesis was accepted and therefore held as tenable. The difference was found 
to be a medium effect size (d = 0.51). 

Conclusions, Implications and Recommendations 
Conclusions: Descriptive 

The students in both CASE and traditional courses spent the majority of their time on- 
task. Students were on-task for an average of more than 60% of the intervals. Elowever, the ratio 
of on-task to off-task behavior was less than 2:1. Students in both groups were found to be off- 
task for more than 40% of the intervals. The majority of this off-task behavior was found to be 
cases of motor activity. The findings about students’ time on-task versus time off-task differ 
from the literature on this subject. Most studies of traditional classrooms indicate a range of 70% 
to 96% time on-task (DuPaul et al., 2004; Elintze & Matthews, 2004; Spanjers, Bums, & Wagner, 
2008). 

Implications: Descriptive 

Shapiro (2011a) describes time on-task as the most significant observable behavior when 
using the BOSS instrument. Given the links between student achievement and engagement rates 
(Greenwood, 1991; Resnick, 2007), the levels of academic engagement could be areas of 
concern. Elowever, caution must be used when evaluating videos of student behavior due to the 
lack of ability to observe the entire classroom situation (Stigler, Gallimore, & Eliebert, 2000). 
Students who did not agree to participate in the study or who were not captured on camera were 
not included in the researcher’s observations. The opportunity to score only the students who 
were recorded could have an effect on the results. The researcher noticed audible instances of on- 
and off-task behavior from students out of view of the camera that might have impacted the 
results of this study. It is also important to reiterate that students can be engaged and off-task in 
the same interval. 

Animal science can be a course with a high amount of integration. If a scientific class 
has engagement of about 60%, could other classes be higher or lower? Also, students might not 
have a strong emotional engagement to the scientific content, which could impact the academic 
engagement. 

Conclusions: Hypothesis Testing 

Statistical analysis indicated the use of the CASE curriculum had a significant impact on 
student academic engagement. While no differences were seen in the amount of time on-task of 
students in CASE and traditional courses, the way in which time was distributed varied. Students 
in CASE courses were found to spend significantly more time actively engaged than students in 
traditional courses. CASE students spent just over 10% more intervals actively engaged. 
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Conversely, students in traditional courses were found to spend significantly more time passively 
engaged in their courses. The difference between the two groups indicated nearly 15% more 
intervals passively engaged for students in traditional courses. When looking at the amount of 
time students spent off-task, no significant differences were found. Students in both groups spent 
nearly the same percentage of intervals in each of the three categories of off-task behavior. 
However, off-task motor activity did have a large practical effect size. 

Implications: Hypothesis Testing 

The differences in engaged time for students is a point of interest. Fredricks et al. (2004) 
reported behavioral engagement measurements such as this are correlated highly with student 
achievement scores. CASE curriculum students did not exhibit more time on-task in this study. 
However, levels of active engagement were higher. These increased levels were possibly a result 
of the use of the CASE curriculum. Carroll’s (1963) Model of School Learning and Shapiro 
(2011a) suggest students who spend more time actively engaged are more likely to learn the 
material and perform better on subsequent assessments of content knowledge. While students’ 
gains in behavioral engagement were found in this study, Velez, Lambert, and Elliot (2012) found 
no changes in CASE students’ cognitive engagement. To date, no studies have been found that 
investigated students’ emotional engagement when using the CASE curriculum. 

An increase in active engagement may lead to opportunities for student off-task behavior. 
Creating an environment with active engagement may require teachers to create more structure 
for students to reduce the opportunities for off-task behavior. A casual observation of the 
classrooms indicated more activities in the CASE classrooms that required students to be out of 
their seats at stations or labs, the activities could lead to students being more off-task motor as 
they moved around the classroom. 

Recommendations for Practitioners 

High School Teachers. Finn and Rock (1997) found that discipline problems and 
behavior issues are reduced with higher levels of engagement. Behavior issues can stem from 
off-task behaviors. While it was found the CASE curriculum can improve the active engagement 
of students, the amount of time students spent on-task was below the levels observed in core 
subject classrooms. It is recommended that agriculture teachers continue to increase the level of 
engagement in their classrooms to reduce the level of off-task behavior. This becomes a 
challenge when scientific curriculums have a large amount of laboratory and activities. Since 
laboratory activities tend to have down time, such as students waiting for results or as one student 
looks in a microscope their partners wait, it is recommended that teachers use their training in 
planning, to anticipate down time and create activities to fill time that could become off-task 
behavior. As Erry and Wood (2011) found, if teachers do not create engagement, students will 
not create the engagement on their own. 
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Teacher Educators. Mulholland and Capello (2006) found that curriculum designed to 
improve teachers’ skills in the classroom had a positive effect on engagement and achievement. 
It is recommended that teacher educators strengthen pre-service teacher instruction on 
engagement and laboratory/activity management. While the total engagement between the two 
groups was not different, the active engagement was higher in the CASE classrooms. The CASE 
teachers had completed in-service training on the CASE curriculum, but this may be due solely to 
the design of the curriculum. It is recommended that teacher educators included engagement in 
the in-service training they conduct. It is also recommended that the developers of the CASE 
Institutes include opportunities for discussions on reducing downtime and increasing active 
engagement. 

Recommendations for Further Research 

Given the small population of this study and the limited generalizability, the study should 
be replicated with a larger population. This will continue to develop the body of knowledge 
surrounding academic engagement of students in those courses. Studies should be conducted to 
investigate the impact of engagement and curriculum on student achievement in agricultural 
education. Other variables that impact achievement, such as cognitive behavior, should be 
studied in relation to student engagement. As students become familiar with inquiry-based 
learning, they adjust to the daily requirements, it is recommended that programs that have 
adopted pathways of inquiry-based classes be studied to better understand the impact that student 
conditioning can have on engagement. 


References 

Appleton, J. J., Christenson, S. L., Kim, D., & Reschly, A. L. (2006). Measuring cognitive and 

psychological engagement: Validation of the student engagement instrument. Journal of 
School Psychology’, 44, 427-445. doi: 10.1016/j jsp.2006.04.002 

Axelson, R. D., & Flick, A. (2011). Defining student engagement. Change: The Magazine of 
Higher Learning, 43(1), 38-43. doi: 10.1080/00091383.2011.533096 

Babad, E. (2005). Nonverbal behavior in education. In J. Harrigan, R. Rosenthal, & K. Scherer 
(Eds.), The new handbook of methods in nonverbal behavior research (pp. 283-311). 
New York, NY: University Press. 

Berliner, D. C. (1980). Using research on teaching for the improvement of classroom practice. 
Theory’ into Practice, 19(4), 302-308. 

Bloom, B. (1974). Time and learning. American Psychologist, 29(9), 682-688. doi: 
10.1037/h0037632 

Bloom, B. S. (1976). Human characteristics and school learning. New York, NY: McGraw-Hill. 

Brich, S., & Ladd, G. (1997). The teacher-child relationship and children’s early school 
adjustment. Journal of School Psychology’, 35, 61-79. doi: 10.1016/S0022- 
4405(96)00029-5 

Caldwell, J.H, Huitt, W. G., & Graeber, A. O. (1982). Time spent in learning: Implications from 
research. The Elementary School Journal, 82(5), 471-480. doi: 10.1086/461282 


Journal of Agricultural Education 


28 


Volume 55, Issue 2, 2014 



Witt et al 


A Comparison of Student Engaged Time... 


Campbell, D. T., & Stanley, J. C. (1963). Experimental and quasi-experimental designs for 
research. Boston: Houghton Mifflin. 

Carroll, J. B. (1963). A model of school learning. Teachers College Record, 64{ 8), 723-733. 

Carroll, J. B. (1989). The Carroll model: A 25-year retrospective and prospective view. 
Educational Researcher, 18(1), 26-31. doi: 10.3102/0013189X018001026 

Clark, D., & Linn, M. C. (2003). Designing for knowledge integration: The impact of 
instructional time. The Journal of the Learning Sciences, 12(4), 451-493. doi: 

10.1207/S 15327809JLS1204_1 

Connell, J. P„ & Wellborn, J. G. (1991). Competence, autonomy, and relatedness: A motivational 
analysis of self-system processes. In M. Gunnar & L. A. Sroufe (Eds.), Minnesota 
Symposium on Child Psychology> (23, pp. 167-216). Chicago, IL: University of Chicago 
Press. 

Connell, J. P., Spencer, M. B., & Aber, J. L. (1994). Educational risk and resilience in African- 
American youth: Context, self, action, and outcomes in school. Child Development, 65, 
493-506. doi: 10.2307/1131398 

Denham, C., & Lieberman, A. (1980). Time to learn: A review of the beginning teacher 
evaluation study. Washington, DC: National Institute of Education. 

Dewalt, M. W., & Rodwell, F. G. (1988). Effects of increased learning time in remedial math and 
science. Spectrum, 6(1), 33-36. 

DuPaul, G. J., Volpe, R. J., Jitendra, A. K., Lutz, J. G., Lorah, K. S., & Gruber, R. (2004). 

Elementary school students with AD/HD: Predictors of academic achievement. Journal 
of School Psychology>, 42, 285-301. 

Erry, R., & Wood, G. (2011). Lessons from a student engagement pilot study. Australian 
Universities Review, 53(1), 21-34. 

Finn, J. D. (1993). School engagement and students at risk. Washington, DC: National Center for 
Education Statistics. 

Finn, J. D., Pannozzo, G. M., & Voelkl, K. E. (1995). Disruptive and inattentive-withdrawn 

behavior and achievement among fourth graders. Elementary> School Journal, 95, 421- 
454. doi: 10.1086/461853 

Finn, J. D., & Rock, D. A. (1997). Academic success among students at risk for school failure. 
Journal of Applied Psychology:, 82, 221-234. doi: 10.1037/0021-9010.82.2.221 

Frank, M. G., Juslin, P. N., & Harrigan, J. A. (2005). Technical issues in recording nonverbal 
behavior. In J. Harrigan, R. Rosenthal, & K. Scherer (Eds.), The new handbook of 
methods in nonverbal behavior research (pp. 283-311). New York, NY: University Press. 

Fredricks, J. A., Blumenfeld, P. C., & Paris, A. H. (2004). School engagement: Potential of the 
concept, state of the evidence. Review of Educational Research, 74(1), 59-109. doi: 
10.3102/00346543074001059 


Journal of Agricultural Education 


29 


Volume 55, Issue 2, 2014 



Witt et al 


A Comparison of Student Engaged Time... 


Gall, M. D., Gall, J. P., & Borg, W. R. (2007). Educational research: An introduction. Boston, 
MA: Pearson Education. 

Goodman, L. (1990). Time and learning in the special education classroom. Albany, NY: State 
University of New York Press. 

Greenwood, C. R. (1991). A longitudinal analysis of time to learn, engagement, and academic 
achievement in urban versus suburban schools. Exceptional Children, 57, 521-535. 

Hintze, J. M., & Matthews, W. J. (2004). The generalizability of systematic direct observations 
across time and setting: A preliminary investigation of the psychometrics of behavioral 
observation. School Psychology: Review, 33(2), 258-270. 

Karweit, N. (1983). Time on task: A research review (Report No. 332). Baltimore, MD: Center 
for social organization of schools. 

Karweit, N., & Slavin, R. E. (1981). Measurement and modeling choices in studies of time and 
learning. American Educational Research Journal, 18(2), 157-171. 

Mautone, J. A., DuPaul, G. J., & Jitendra, A. K. (2005). The effects of computer-assisted 

instruction on the mathematics performance and classroom behavior of children with 
ADHD. Journal of Attention Disorders, 9(1), 301-312. doi: 10.1177/1087054705278832 

McQuillan, K., DuPaul, G. J., Shapiro, E. S., & Cole, C. L. (1996). Classroom performance of 
students with serious emotional disturbance: A comparative study of evaluation methods 
for behavior management. Journal of Emotional and Behavioral Disorders, 4(3), 162- 
170. 

Meece, J. L., Blumenfeld, P. C., & Hoyle, R. H. (1988). Students’ goal orientations and cognitive 
engagement in classroom activities. Journal of Educational Psychology:, 80(4), 514-523. 

Mulholland, R., & Cepello, M. (2006). What teacher candidates need to know about academic 
learning time. International Journal of Special Education 21(2), 63-73. 

National Education Commission on Time and Learning (NECTL). (1994). National education 
commission on time and learning briefing paper. Washington, DC. 

New York City, Mayoral Task Force on Career and Technical Education Innovation. (2008). 
Next-Generation Career and Technical Education in New York City. Retrieved from 
http://schools.nyc.gov/NR/rdonlyres/DAC6F4E0-DlA6-4B0A-809C- 
CB9DA61988A8/72746/NY CCTE7231owres.pdf 

Newcomb, L. H., McCracken, J. D., & Warmbrod, J. R. (1993). Methods of teaching agriculture. 
(2nd ed.). Danville, IL: Interstate Publishers. 

Parr, B., Edwards, M. C., & Leising, J. G. (2009). Selected effects of a curriculum integration 
intervention on the mathematics performance of secondary students enrolled in an 
agricultural power and technology course: An experimental study. Journal of 
Agricultural Education, 59(1), 57-69. doi: 10.5032/jae.2009.01057 

Resnick, L. B. (2007). Time to learn. Research Points, 5(2), 1-4. 


Journal of Agricultural Education 


30 


Volume 55, Issue 2, 2014 



Witt et al 


A Comparison of Student Engaged Time... 


Shapiro, E. S. (2011a). Academic skills problems: direct assessment and intervention. (4th ed.). 
New York: the Guilford Press. 

Shapiro, E. S. (2011b). Academic skills problems workbook. (4th ed.). New York: The Guilford 
Press. 

Skinner, E. A., & Belmont, M. J. (1993). Motivation in the classroom: Reciprocal effects of 

teacher behavior and student engagement across the school year. Journal of Educational 
Psychology>, 55(4), 571-581. doi: 10.1037/0022-0663.85.4.571 

Skinner, E. A., Wellborn, J. G., & Connell, J. P. (1990). What it takes to do well in school and 

whether I’ve got it: A process model of perceived control and children’s engagement and 
achievement in school. Journal of Educational Psychology>, 52(1), 22-32. doi: 
10.1037/0022-0663.82.1.22 

Slavin, R. E. (2009). Educational psychology>: Theory> and practice (9th ed.). Upper Saddle River, 
New Jersey: Pearson 

Spanjers, D. M., Bums, M. K., & Wagner, A. R. (2008). Systematic direct observation of time on 
task as a measure of student engagement. Assessment for Effective Intervention, 33(2), 
120-126. doi: 10.1177/1534508407311407 

Stanley, S. O., & Greenwood, C. R. (1981). Code for instructional structure and student 

academic response (CISSAR): Observer’s manual. Kansas City: University of Kansas, 
Bureau of Child Research, Juniper Gardens Children’s Project. 

Stigler, J. W., Gallimore, R., & Hiebert, J. (2000). Using video surveys to compare classrooms 
and teaching across cultures: Examples and lessons from the TIMSS video studies. 
Educational Psychologist, 55(2), 87-100. 

Taylor, L„ & Parsons, J. (2011). Improving student engagement. Current Issues in Education, 
14(1), 1-33. 

Thalheimer, W., & Cook, S. (2002). How to calculate effects sizes from published research: A 
simplified methodology’. Retrieved from http://www.workleaming.com/ 
Catalog/index.htm. 

Velez, J. J., Lambert, M. D., & Elliot, K. M. (2012). High school students’ perceptions of their 
yearlong CASE experience. Proceedings of the American Association of Agricultural 
Education, Western Region, 1-16. 

Vile Junod, R. E., DuPaul, G. J., Jitendra, A. K., Volpe, R. J., & Cleary, K. S. (2006). Classroom 
observations of students with and without ADHD: Differences across types of 
engagement. Journal of School Psychology>, 44, 87-104. doi: 10.1016/j .jsp.2005.12.004 

Volpe, R. J., DiPerna, J. C., Hintze, J. M., & Shapiro, E. S. (2005). Observing students in 

classroom settings: A review of seven coding schemes. School Psychology> Review, 34(4), 
454-474. 


Journal of Agricultural Education 


31 


Volume 55, Issue 2, 2014 



Witt et al 


A Comparison of Student Engaged Time... 


Wiley, D. E., & Hamischfeger, A. (1974). Explosion of a myth: Quantity of schooling and 

exposure to instruction, major education vehicles. Educational Researcher, 3(4), 7-12. 


doi: 10.2307/1175988 


Journal of Agricultural Education 


32 


Volume 55, Issue 2, 2014 



